A counterview of ‘An investigation of the false discovery rate and the misinterpretation of p-values’ by Colquhoun (2014)

A counterview of 'An investigation of the false discovery rate and the misinterpretation of p-values' by Colquhoun (2014) In commenting on the instructive, comprehensive and entertainingly written article by Prof. Colquhoun (hereinafter referred to as 'the author'), we state unequivocally that we have no truck with its motivation. Indeed, we too often find the Fisherian approach troubling. Nor do we wish to become involved in the relative merits of 'Bayesian' versus 'Fisherian' methods. Rather we wish to focus on the author's underlying model, reminding the reader that the output of any mathematical model is only as good as its input parameters. In this regard, we find the numeric value of 0.1 for the parameter describing 'the probability that the putative effect is real' to be wholly unrealistic for divining the appropriate p-value to be used as the basis for deciding whether the outcome of an experiment provides evidence 'for' or 'against' rejection of the null hypothesis. We readily admit that we have no more idea than does the author regarding the true value of the 'prevalence' parameter for experimental science, so we have adopted three distinct approaches to estimate it. (i) First, and with reference to the author's charmingly apposite introductory quote from George Elliot's Middlemarch, we state our 'gut instinct' estimate to be 'greater than 50%'. (ii) Second, and widening the scope, we have canvassed the senior investigators in our Department of Physiology for their personal estimates of the fraction of times that their explicit, experimentally testable hypotheses have proven to be supported by experimental results. We are aware of the somewhat circular logic of this undertaking because, in each case, 'classical' hypothesis testing underlies the 'guesstimates'. Nevertheless, we consider that well-informed scientists can do better than a flip-of-the-coin (and certainly better than the roll of a decahedral die) in guessing the pathway along which truth lies. (iii) Finally, we have examined the 2015 The Authors. Published by the Royal Society under the terms of the Creative Commons Attribution License